Vehicle control device, vehicle control method, and storage medium

ABSTRACT

A vehicle control device includes an imaging unit that images surroundings of an own-vehicle, a detector that detects a state of an external appearance of an oncoming vehicle facing the own-vehicle in a tunnel on the basis of an image captured by the imaging unit, and a determiner that determines whether or not the state of the external appearance of the oncoming vehicle satisfies a predetermined condition on the basis of the state of the external appearance of the oncoming vehicle detected by the detector and determines that the outside of the tunnel toward which the own-vehicle is traveling has bad weather if the state of the external appearance of the oncoming vehicle satisfies the predetermined condition.

CROSS-REFERENCE TO RELATED APPLICATION

Priority is claimed on Japanese Patent Application No. 2017-226774,filed Nov. 27, 2017, the content of which is incorporated herein byreference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a vehicle control device, a vehiclecontrol method, and a storage medium.

Description of Related Art

When driving a vehicle, it may pass through a tunnel. If the tunnel islong, the weather may change before entering and after exiting thetunnel. Therefore, the weather may be bad, such as rain and snow, nearthe exit although it was good near the entrance, which may causeinstability in driving. Thus, when passing through a tunnel, the driverof a vehicle collects weather information near the exit through a radioor the like and drives carefully.

A technology in which a wiper is activated upon determining that snow isfalling as a result of collecting weather information throughcommunication is known in the related art (see, for example, JapaneseUnexamined Patent Application, First Publication No. 2014-15164).

SUMMARY OF THE INVENTION

However, weather information near the exit may fail to be collected whenradio waves are bad in the tunnel or Internet communication environmentsare bad.

Aspects of the present invention have been made in view of suchcircumstances and it is an object of the present invention to provide avehicle control device, a vehicle control method, and a storage mediumthat can perform driving control in consideration of the weather on theopposite side of the exit of the tunnel.

A vehicle control device, a vehicle control method, and a storage mediumaccording to the present invention adopt the following configurations.

(1) A vehicle control device according to one aspect of the presentinvention includes an imaging unit configured to image surroundings ofan own-vehicle, a detector configured to detect a state of an externalappearance of an oncoming vehicle facing the own-vehicle in a tunnel onthe basis of an image captured by the imaging unit, and a determinerconfigured to determine whether or not the state of the externalappearance of the oncoming vehicle satisfies a predetermined conditionon the basis of the state of the external appearance of the oncomingvehicle detected by the detector and to determine that the outside ofthe tunnel toward which the own-vehicle is traveling has bad weather ifthe state of the external appearance of the oncoming vehicle satisfiesthe predetermined condition.

(2) In the above aspect (1), the determiner is configured to determinethat the outside of the tunnel toward which the own-vehicle is travelinghas bad weather if the detector detects that snow is attached to theoncoming vehicle.

(3) In the above aspect (1), the determiner is configured to determinethat the outside of the tunnel toward which the own-vehicle is travelinghas bad weather if the detector detects that a wiper of the oncomingvehicle is operating.

(4) In the above aspect (1), the detector is configured to detect astate of a road surface of an oncoming lane in which the oncomingvehicle is present on the basis of an image captured by the imagingunit, and the determiner is configured to determine that the outside ofthe tunnel toward which the own-vehicle is traveling has bad weather ifthe detector detects that the road surface of the oncoming lane is wet.

(5) In the above aspect (1), the determiner is configured to compare aluminance value of an image captured near an entrance of the tunnel bythe imaging unit and a luminance value of an image captured near an exitof the tunnel by the imaging unit and to determine that the outside ofthe tunnel toward which the own-vehicle is traveling has bad weather onthe basis of a result of the comparison.

(6) In the above aspect (1), the vehicle control device further includesa driving controller configured to control one or both of steering oracceleration/deceleration of the own-vehicle, wherein the drivingcontroller is configured to stop control if the determiner determinesthat the outside of the tunnel toward which the own-vehicle is travelinghas bad weather.

(7) In the above aspect (1), the vehicle control device further includesa driving controller configured to control one or both of steering oracceleration/deceleration of the own-vehicle, wherein the drivingcontroller is configured to decelerate the own-vehicle if the determinerdetermines that the outside of the tunnel toward which the own-vehicleis traveling has bad weather.

(8) A vehicle control method according to one aspect of the presentinvention is performed by an in-vehicle computer mounted in anown-vehicle, and includes the in-vehicle computer detecting a state ofan external appearance of an oncoming vehicle facing the own-vehicle ina tunnel on the basis of an image captured by an imaging unit configuredto image surroundings of the own-vehicle, determining whether or not thestate of the external appearance of the oncoming vehicle satisfies apredetermined condition on the basis of the detected state of theexternal appearance of the oncoming vehicle, and determining that theoutside of the tunnel toward which the own-vehicle is traveling has badweather if the state of the external appearance of the oncoming vehiclesatisfies the predetermined condition.

(9) A storage medium according to one aspect of the present invention isa computer readable non-transitory storage medium storing a programcausing an in-vehicle computer mounted in an own-vehicle having animaging unit configured to image surroundings of the own-vehicle todetect a state of an external appearance of an oncoming vehicle facingthe own-vehicle in a tunnel on the basis of an image captured by theimaging unit, determine whether or not the state of the externalappearance of the oncoming vehicle satisfies a predetermined conditionon the basis of the detected state of the external appearance of theoncoming vehicle, and determine that the outside of the tunnel towardwhich the own-vehicle is traveling has bad weather if the state of theexternal appearance of the oncoming vehicle satisfies the predeterminedcondition.

According to the above aspects (1) to (9), regardless of thecommunication environments, it is possible to perform driving control inconsideration of the weather on the opposite side of the exit of thetunnel.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a configuration diagram of a vehicle system using a vehiclecontrol device according to an embodiment.

FIG. 2 is a functional configuration diagram of a first controller and asecond controller.

FIG. 3 is a diagram showing how a target trajectory is generated on thebasis of a recommended lane.

FIG. 4 is an example of an image obtained by imaging a scene in which awhite object is on a roof of an oncoming vehicle.

FIG. 5 is an example of an image obtained by imaging a scene in which awiper is moving on a windshield of an oncoming vehicle.

FIG. 6 is an example of an image obtained by imaging a scene in which aroad surface of an oncoming lane is wet.

FIG. 7 is an example of an image obtained by imaging a scene near theentrance of a tunnel.

FIG. 8 is an example of an image obtained by imaging a scene near theexit of the tunnel.

FIG. 9 is a flowchart showing an example of a flow of a processperformed by a first controller.

FIG. 10 is a flowchart showing an example of a flow of a firstdetermination process performed by a determiner.

FIG. 11 is a flowchart showing an example of a flow of a seconddetermination process performed by the determiner.

FIG. 12 is a flowchart showing an example of a flow of a thirddetermination process performed by the determiner.

FIG. 13 is a flowchart showing an example of a flow of a fourthdetermination process performed by the determiner.

FIG. 14 is a flowchart showing an example of a flow of a fifthdetermination process performed by the determiner.

FIG. 15 is a configuration diagram of a vehicle system using the vehiclecontrol device according to an embodiment.

FIG. 16 is a diagram showing an example of a hardware configuration ofthe vehicle control device according to an embodiment.

DETAILED DESCRIPTION OF THE INVENTION First Embodiment

Hereinafter, embodiments of a vehicle control device, a vehicle controlmethod, and a storage medium of the present invention will be describedwith reference to the drawings. The following description will be givenwith reference to the case in which left-hand traffic laws are applied,but the terms “left” and “right” only need to be reversely read whenright-hand traffic laws are applied.

[Overall Configuration]

FIG. 1 is a configuration diagram of a vehicle system 1 using a vehiclecontrol device according to a first embodiment. A vehicle in which thevehicle system 1 is mounted is, for example, a vehicle such as atwo-wheeled vehicle, a three-wheeled vehicle, or a four-wheeled vehicle,and a driving source thereof is an internal combustion engine such as adiesel engine or a gasoline engine, an electric motor, or a combinationthereof. When an electric motor is provided, the electric motor operatesusing electric power generated by a generator connected to the internalcombustion engine or using discharge power of a secondary battery or afuel cell.

The vehicle system 1 includes, for example, a camera 10, a radar device12, a finder 14, an object recognition device 16, a communication device20, a human machine interface (HMI) 30, vehicle sensors 40, a navigationdevice 50, a map positioning unit (MPU) 60, driving operators 80, anautomated driving control device 100, a travel driving force outputdevice 200, a brake device 210, and a steering device 220. These devicesor apparatuses are connected to each other by a multiplex communicationline or a serial communication line such as a controller area network(CAN) communication line, a wireless communication network, or the like.The components shown in FIG. 1 are merely an example and some of thecomponents may be omitted or other components may be added.

The camera 10 is, for example, a digital camera using a solid-stateimaging device such as a charge coupled device (CCD) or complementarymetal oxide semiconductor (CMOS) image sensor. One or a plurality ofcameras 10 are attached to the vehicle in which the vehicle system 1 ismounted (hereinafter referred to as an own-vehicle M) at arbitrarylocations. For imaging the area in front of the vehicle, a camera 10 isattached to an upper portion of a front windshield, a rear surface of arearview mirror, or the like. For example, the camera 10 repeats imagingof the surroundings of the own-vehicle M at regular intervals. Thecamera 10 may also be a stereo camera.

The radar device 12 radiates radio waves such as millimeter waves aroundthe own-vehicle M and detects radio waves reflected by an object(reflected waves) to detect at least the position (distance andorientation) of the object. One or a plurality of radar devices 12 maybe attached to the own-vehicle M at arbitrary locations. The radardevice 12 may detect the position and velocity of an object using afrequency modulated continuous wave (FM-CW) method.

The finder 14 is a light detection and ranging (LIDAR) finder. Thefinder 14 illuminates the surroundings of the own-vehicle M with lightand measures scattered light. The finder 14 detects the distance to atarget on the basis of a period of time from when light is emitted towhen light is received. The light illuminated is, for example, pulsedlaser light. One or a plurality of finders 14 may be attached to theown-vehicle M at arbitrary locations.

The object recognition device 16 performs a sensor fusion process onresults of detection by some or all of the camera 10, the radar device12, and the finder 14 to recognize the position, type, speed, or thelike of the object. The object recognition device 16 outputs therecognition result to the automated driving control device 100. Asnecessary, the object recognition device 16 may output detection resultsof the camera 10, the radar device 12 and the finder 14 to the automateddriving control device 100 as they are. A speed acquirer may include theradar device 12.

For example, the communication device 20 communicates with othervehicles near the own-vehicle M using a cellular network, a Wi-Finetwork, Bluetooth (registered trademark), dedicated short rangecommunication (DSRC) or the like or communicates with various serverdevices via wireless base stations.

The HMI 30 presents various types of information to an occupant in theown-vehicle M and receives an input operation from the occupant. The HMI30 includes various display devices, a speaker, a buzzer, a touch panel,switches, keys, or the like.

The vehicle sensors 40 include a vehicle speed sensor that detects thespeed of the own-vehicle M, an acceleration sensor that detects theacceleration thereof, a yaw rate sensor that detects an angular speedthereof about the vertical axis, an orientation sensor that detects theorientation of the own-vehicle M, or the like.

The navigation device 50 includes, for example, a global navigationsatellite system (GNSS) receiver 51, a navigation HMI 52, and a routedeterminer 53 and holds first map information 54 in a storage devicesuch as a hard disk drive (HDD) or a flash memory. The GNSS receiver 51specifies the position of the own-vehicle M on the basis of signalsreceived from GNSS satellites. The position of the own-vehicle M mayalso be specified or supplemented by an inertial navigation system (INS)using the output of the vehicle sensors 40. The navigation HMI 52includes a display device, a speaker, a touch panel, a key, or the like.The navigation HMI 52 may be partly or wholly shared with the HMI 30described above. For example, the route determiner 53 determines a routefrom the position of the own-vehicle M specified by the GNSS receiver 51(or an arbitrary input position) to a destination input by the occupant(hereinafter referred to as an on-map route) using the navigation HMI 52by referring to the first map information 54. The first map information54 is, for example, information representing shapes of roads by linksindicating roads and nodes connected by the links. The first mapinformation 54 may include curvatures of roads, point of interest (POI)information, or the like. The on-map route determined by the routedeterminer 53 is output to the MPU 60. The navigation device 50 may alsoperform route guidance using the navigation HMI 52 on the basis of theon-map route determined by the route determiner 53. The navigationdevice 50 may be realized, for example, by a function of a terminaldevice such as a smartphone or a tablet possessed by the occupant. Thenavigation device 50 may also transmit the current position and thedestination to a navigation server via the communication device 20 andacquire an on-map route returned from the navigation server.

The MPU 60 functions, for example, as a recommended lane determiner 61and holds second map information 62 in a storage device such as an HDDor a flash memory. The recommended lane determiner 61 divides the routeprovided from the navigation device 50 into a plurality of blocks (forexample, into blocks each 100 meters long in the direction in which thevehicle travels) and determines a recommended lane for each block byreferring to the second map information 62. The recommended lanedeterminer 61 determines the recommended lane such that it is given aposition in a lane order counted from the leftmost lane. When there is abranch point, a merge point, or the like on the route, the recommendedlane determiner 61 determines a recommended lane such that theown-vehicle M can travel on a reasonable route for proceeding to thebranch destination.

The second map information 62 is map information with higher accuracythan the first map information 54. The second map information 62includes, for example, information of the centers of lanes orinformation of the boundaries of lanes. The second map information 62may also include road information, traffic regulation information,address information (addresses/postal codes), facility information,telephone number information, or the like. The second map information 62may be updated as needed by accessing another device using thecommunication device 20.

The driving operators 80 include, for example, an accelerator pedal, abrake pedal, a shift lever, a steering wheel, a different shapedsteering member, a joystick, and other operators. Sensors for detectingthe amounts of operation or the presence or absence of operation areattached to the driving operators 80. Results of the detection areoutput to the automated driving control device 100 or at least one orall of the travel driving force output device 200, the brake device 210,and the steering device 220.

The automated driving control device 100 includes, for example, a firstcontroller 120 and a second controller 160. Each of the first controller120 and the second controller 160 is realized, for example, by ahardware processor such as a central processing unit (CPU) executing aprogram (software). Some or all of these components may be realized byhardware (including circuitry) such as large scale integration (LSI), anapplication specific integrated circuit (ASIC), a field-programmablegate array (FPGA), or a graphics processing unit (GPU) or may berealized by hardware and software in cooperation. The program may bestored in a storage device such as a hard disk drive (HDD) or a flashmemory in advance or may be stored in a detachable storage medium suchas a DVD or a CD-ROM and then installed in the storage device byinserting the storage medium into a drive device.

FIG. 2 is a functional configuration diagram of the first controller 120and the second controller 160. The first controller 120 includes, forexample, a recognizer 130 and a behavior plan generator 150. Forexample, the first controller 120 realizes a function based onartificial intelligence (AI) and a function based on a previously givenmodel in parallel. For example, the function of “recognizing anintersection” is realized by performing recognition of an intersectionthrough deep learning or the like and recognition based on previouslygiven conditions (presence of a signal, a road sign, or the like forwhich pattern matching is possible) in parallel and evaluating bothcomprehensively through scoring. This guarantees the reliability ofautomated driving.

The recognizer 130 recognizes states of an object near the own-vehicle Msuch as the position, speed and acceleration thereof on the basis ofinformation input from the camera 10, the radar device 12, and thefinder 14 via the object recognition device 16. The position of theobject is recognized, for example, as a position in an absolutecoordinate system whose origin is a representative point of theown-vehicle M (such as the center of gravity or the center of a driveaxis thereof), and used for control. The position of the object may berepresented by a representative point of the object such as the centerof gravity or a corner thereof or may be represented by an expressedregion. The “states” of the object may include an acceleration or jerkof the object or a “behavior state” thereof (for example, whether or notthe object is changing or is going to change lanes). The recognizer 130recognizes the shape of a curve that the own-vehicle M is about to passon the basis of an image captured by the camera 10. The recognizer 130converts the shape of the curve from the captured image of the camera 10into a real plane and outputs information expressed, for example, usingtwo-dimensional point sequence information or a model equivalent theretoto the behavior plan generator 150 as information indicating the shapeof the curve.

The recognizer 130 recognizes, for example, a (traveling) lane in whichthe own-vehicle M is traveling. A recognition result of the laneindicates, for example, which lane the own-vehicle M is traveling inamong a plurality of lanes in the same travel direction. When the numberof lanes is one, this fact may be a recognition result. For example, therecognizer 130 recognizes the traveling lane, for example, by comparinga pattern of road lane lines (for example, an arrangement of solid andbroken lines) obtained from the second map information 62 with a patternof road lane lines near the own-vehicle M recognized from an imagecaptured by the camera 10. The recognizer 130 may recognize thetraveling lane by recognizing travel boundaries (road boundaries)including road lane lines, road shoulders, curbs, a median strip, guardrails, or the like, without being limited to road lane lines. Thisrecognition may be performed taking into consideration a position of theown-vehicle M acquired from the navigation device 50 or a result ofprocessing by the INS. The recognizer 130 recognizes temporary stoplines, obstacles, red lights, toll gates, and other road phenomena.

When recognizing the traveling lane, the recognizer 130 recognizes theposition or attitude of the own-vehicle M with respect to the travelinglane. For example, the recognizer 130 may recognize both a deviationfrom the lane center of the reference point of the own-vehicle M and anangle formed by the travel direction of the own-vehicle M relative to anextension line of the lane center as the relative position and attitudeof the own-vehicle M with respect to the traveling lane. Alternatively,the recognizer 130 may recognize the position of the reference point ofthe own-vehicle M with respect to one of the sides of the traveling lane(a road lane line or a road boundary) or the like as the relativeposition of the own-vehicle M with respect to the traveling lane.

In the above recognition process, the recognizer 130 may derive theaccuracy of recognition and output it as recognition accuracyinformation to the behavior plan generator 150. For example, therecognizer 130 generates recognition accuracy information on the basisof the frequency of recognition of road lane lines in a certain period.

The recognizer 130 includes a tunnel recognizer 140. The tunnelrecognizer 140 includes a timing determiner 142, a state detector 144,and a determiner 146. These components will be described later.

The behavior plan generator 150 determines events which are to besequentially performed in the automated driving, basically such that theown-vehicle M travels in the recommended lane determined by therecommended lane determiner 61 and copes with situations occurring nearthe own-vehicle M. Examples of the events include a constant-speedtravel event which is an event of traveling in the same traveling laneat a constant speed, a following travel event which is an event offollowing a preceding vehicle, an overtaking event which is an event ofovertaking a preceding vehicle, an avoidance event which is an event ofperforming braking and/or steering to avoid approaching an obstacle, acurve traveling event which is an event of traveling on a curve, apassing event which is an event of passing through a predetermined pointsuch as an intersection, a pedestrian crossing, or a railroad crossing,a lane change event, a merging event, a branching event, an automaticstop event, and a takeover event which is an event of ending automateddriving and switching to manual driving.

The behavior plan generator 150 generates a target trajectory alongwhich the own-vehicle M will travel in the future according to anactivated event. Details of each functional unit will be describedlater. The target trajectory includes, for example, a speed element. Thetarget trajectory is expressed, for example, by an arrangement of points(trajectory points) which are to be reached by the own-vehicle M inorder. The trajectory points are points to be reached by the own-vehicleM at intervals of a predetermined travel distance (for example, atintervals of several meters) along the road. Apart from this, a targetspeed and a target acceleration for each predetermined sampling time(for example, every several tenths of a second) are determined as a partof the target trajectory. The trajectory points may be respectivepositions of the predetermined sampling times which the own-vehicle M isto reach at the corresponding sampling times. In this case, informationon the target speed or the target acceleration is represented with theinterval between the trajectory points.

FIG. 3 is a diagram showing how a target trajectory is generated on thebasis of a recommended lane. As shown, the recommended lane is set to beconvenient for traveling along the route to the destination. When theown-vehicle M approaches a predetermined distance (which may bedetermined according to the types of events) before a point forswitching to the recommended lane, the behavior plan generator 150activates a passing event, a lane change event, a branching event, amerging event, or the like. When it becomes necessary to avoid anobstacle during execution of each event, an avoidance trajectory isgenerated as shown.

The second controller 160 controls the travel driving force outputdevice 200, the brake device 210, and the steering device 220 such thatthe own-vehicle M passes through the target trajectory generated by thebehavior plan generator 150 at scheduled times.

Returning to FIG. 2, the second controller 160 includes, for example, anacquirer 162, a speed controller 164, and a steering controller 166. Theacquirer 162 acquires information on the target trajectory (trajectorypoints) generated by the behavior plan generator 150 and stores it in amemory (not shown). The speed controller 164 controls the travel drivingforce output device 200 or the brake device 210 on the basis of thespeed element included in the target trajectory stored in the memory.The steering controller 166 controls the steering device 220 accordingto the degree of curvature of the target trajectory stored in thememory. The processing of the speed controller 164 and the steeringcontroller 166 is realized, for example, by a combination of feedforwardcontrol and feedback control. As one example, the steering controller166 performs the processing by combining feedforward control accordingto the curvature of the road ahead of the own-vehicle M and feedbackcontrol based on deviation from the target trajectory.

The travel driving force output device 200 outputs a travel drivingforce (torque) required for the vehicle to travel to driving wheels. Thetravel driving force output device 200 includes, for example, acombination of an internal combustion engine, an electric motor, atransmission, and the like and an ECU that controls them. The ECUcontrols the above constituent elements according to information inputfrom the second controller 160 or information input from the drivingoperators 80.

The brake device 210 includes, for example, a brake caliper, a cylinderthat transmits hydraulic pressure to the brake caliper, an electricmotor that generates hydraulic pressure in the cylinder, and a brakeECU. The brake ECU controls the electric motor according to informationinput from the second controller 160 or information input from thedriving operators 80 such that a brake torque corresponding to a brakingoperation is output to each wheel. The brake device 210 may include, asa backup, a mechanism for transferring a hydraulic pressure generated byan operation of the brake pedal included in the driving operators 80 tothe cylinder via a master cylinder. The brake device 210 is not limitedto that configured as described above and may be an electronicallycontrolled hydraulic brake device that controls an actuator according toinformation input from the second controller 160 and transmits thehydraulic pressure of the master cylinder to the cylinder.

The steering device 220 includes, for example, a steering ECU and anelectric motor. The electric motor, for example, applies a force to arack-and-pinion mechanism to change the direction of steering wheels.The steering ECU drives the electric motor according to informationinput from the second controller 160 or information input from thedriving operators 80 to change the direction of the steering wheels.

Next, each of the components of the tunnel recognizer 140 included inthe recognizer 130 will be described in detail.

The timing determiner 142 determines whether or not the own-vehicle M istraveling near the entrance or exit of a tunnel. For example, the timingdeterminer 142 compares the position and route of the own-vehicle M withthe second map information 62 and determines that the own-vehicle istraveling near the entrance or exit of a tunnel when the own-vehicle Mhas reached a predetermined distance before the entrance or exit of atunnel. The timing determiner 142 may recognize the shape of theentrance or exit of the tunnel present ahead in the travel direction ofthe own-vehicle M on the basis of an image captured by the camera 10using a method such as pattern matching and determine whether or not theown-vehicle M has reached a predetermined distance before the entranceor exit of the tunnel on the basis of the size or the like of therecognized tunnel. Upon determining that the own-vehicle M is travelingnear the entrance or exit, the timing determiner 142 outputs thedetermination result to the state detector 144.

The state detector 144 detects the states of an oncoming vehicle mcorresponding to the own-vehicle M in the tunnel on the basis of theimage captured by the camera 10. For example, the state detector 144detects the states of the oncoming vehicle m by analyzing images offrames before and after the timing determined by the timing determiner142 out of images captured by the camera 10. The states of the oncomingvehicle m detected by the state detector 144 include, for example, astate in which snow is on the roof of the oncoming vehicle m, a state inwhich the vehicle body of the oncoming vehicle m is wet, or a state inwhich a wiper is moving on the windshield of the oncoming vehicle m.

The state detector 144 detects the states of the oncoming vehicle m, forexample, using a machine learning method such as deep learning. Thestate detector 144 may detect the states of the oncoming vehicle mthrough a modeling method such as pattern matching or may perform themachine learning method and the modeling method in parallel. Upondetecting a state of the oncoming vehicle m predetermined as adetermination target, the state detector 144 outputs informationindicating the detection of the state to the determiner 146.

FIGS. 4 and 5 are examples of images captured by the camera 10. FIG. 4is an example of an image 301 obtained by imaging a scene in which snowSN is on the roof of the oncoming vehicle m. FIG. 5 is an example of animage 302 obtained by imaging a scene in which the wiper WP is moving onthe windshield of the oncoming vehicle m. The state detector 144 detectsa state in which snow is on the roof of the oncoming vehicle m on thebasis of the image 301 using the method described above. The statedetector 144 detects a state in which the wiper of the oncoming vehiclem is operating on the basis of the image 302 using the method describedabove.

The state detector 144 may detect the state of the road surface of anoncoming lane in which the oncoming vehicle m is present on the basis ofthe image captured by the camera 10. For example, the state detector 144derives an average luminance value Bv11 of a region corresponding to theown lane in which the own-vehicle M is traveling and an averageluminance value Bv12 of a region corresponding to the oncoming lane inthe captured image, compares the derived average luminance value Bv11 ofthe own lane and the derived average luminance value Bv12 of the regioncorresponding to the oncoming lane, and detects a state in which theroad surface of the oncoming lane is wet or frozen if the differencebetween the luminance values is equal to or greater than a thresholdvalue. FIG. 6 is an example of an image 303 obtained by imaging a scenein which the road surface of the oncoming lane is wet. When the image303 as shown in FIG. 6 is captured by the camera 10, the state detector144 outputs information indicating that the oncoming lane is wet to thedeterminer 146.

The state detector 144 may detect the state of the opposite side of theexit of the tunnel toward which the own-vehicle M is traveling on thebasis of the image captured by the camera 10. For example, the statedetector 144 detects that the brightness of the opposite side of theexit of the tunnel is lower than the brightness before the entrance ofthe tunnel on the basis of the difference between an average luminancevalue of a predetermined region of an image captured near the entranceof the tunnel and an average luminance value of a predetermined regionof an image captured near the exit of the tunnel.

Here, an example in which the state of the opposite side of the exit ofthe tunnel is detected on the basis of the luminance difference betweenpredetermined regions of images captured near the entrance and exit ofthe tunnel will be described with reference to FIGS. 7 and 8. FIG. 7 isan example of an image 304 obtained by imaging a scene near the entranceof the tunnel. FIG. 8 is an example of an image 305 obtained by imaginga scene near the exit of the tunnel. The image 304 (or the image 305)is, for example, an image captured at the timing when the own-vehicle Marrives a predetermined distance before the entrance of the tunnel (or apredetermined distance before the exit of the tunnel). The timingdeterminer 142 may acquire an image at a predetermined distance beforethe entrance of the tunnel (or at a predetermined distance before theexit of the tunnel) according to the proportion of the size or shape ofthe tunnel in the entire image.

First, the state detector 144 discriminates between an internal imageregion 304 a obtained by imaging the opposite side of the entrance ofthe tunnel and an external image region 304 b obtained by imaging theother landscape around the tunnel in the image 304. For example, thestate detector 144 derives luminance values of all pixels of the image304 and acquires a boundary line where the difference between luminancevalues of adjacent pixels is equal to or greater than a predeterminedvalue. The state detector 144 recognizes one of the regions divided bythis boundary line having a lower average luminance value as theinternal image region 304 a and recognizes the other having a higheraverage luminance value as the external image region 304 b. Then, thestate detector 144 acquires an average luminance value Bv21 of theexternal image region 304 b. The state detector 144 discriminatesbetween an external image region 305 a obtained by imaging the landscapeof the opposite side of the exit of the tunnel and an internal imageregion 305 b, other than the external image region 305 a, obtained byimaging the inside of the tunnel in the image 305. For example, thestate detector 144 derives luminance values of all pixels of the image305 and acquires a boundary line where the difference between luminancevalues of adjacent pixels is equal to or greater than a predeterminedvalue. The state detector 144 recognizes one of the regions divided bythis boundary line having a higher average luminance value as anexternal image region 305 a and recognizes the other having a loweraverage luminance value as an internal image region 305 b. Then, thestate detector 144 acquires an average luminance value Bv22 of theexternal image region 305 a. The state detector 144 compares the derivedluminance average value Bv21 and the derived luminance average valueBv22 and outputs information indicating that the difference between thetwo values is equal to or greater than a threshold value to thedeterminer 146 if the difference is equal to or greater than thethreshold value. The state detector 144 may compare an average luminancevalue of the entire image near the entrance of the tunnel and an averageluminance value of the entire image near the exit of the tunnel, notlimited to comparing the luminance values of parts of the images.

On the basis of the detection results of the state detector 144 (thestate of the oncoming vehicle m, the state of the oncoming lane, thestate of the opposite side of the exit of the tunnel, or the like), thedeterminer 146 determines whether or not the outside of the tunneltoward which the own-vehicle M is traveling has bad weather. Thedeterminer 146 determines that the outside of the tunnel toward whichthe own-vehicle M is traveling has bad weather, for example, when one ormore of condition (A) that snow is attached to the oncoming vehicle m,condition (B) that the oncoming vehicle m is wet, or condition (C) thatthe wiper of the oncoming vehicle m is operating are satisfied. Thedeterminer 146 may derive the certainty of the determination result onthe basis of the detection result of the state detector 144 and mayoutput the derived certainty to a bad weather controller 151. Forexample, the determiner 146 derives a point corresponding to the numberof detected states among the states (A) to (C) and outputs the derivedpoint as the certainty to the bad weather controller 151.

When it is detected that the road surface of the oncoming lane is wet orfrozen, the determiner 146 determines that the outside of the tunneltoward which the own-vehicle M is traveling has bad weather. Thedeterminer 146 may derive the certainty according to the length of thewet or frozen road surface and may output the derived certainty to thebad weather controller 151. For example, the certainty may be determinedaccording to the length over which the road surface continues to bedetected as being wet or frozen or may be determined according to thedistance to the exit of the tunnel from a position at which the roadsurface is first detected as being wet or frozen.

Further, the determiner 146 determines that the outside of the tunneltoward which the own-vehicle M is traveling has bad weather when thedifference between the average luminance value of an external imageregion in an image captured near the entrance of the tunnel and theaverage luminance value of an external image region in an image capturednear the exit of the tunnel is equal to or greater than a thresholdvalue (that is, when the state detector 144 has detected a state inwhich the brightness of the opposite side of the exit of the tunnel islower than the brightness before the entrance of the tunnel). Thedeterminer 146 may derive the certainty according to the differencebetween the average luminance values and may output the derivedcertainty to the bad weather controller 151. For example, the determiner146 makes the certainty higher when the difference between the averageluminance values is large as compared to when the difference between theaverage luminance values is small.

The determiner 146 may increase the certainty of the determinationresult by combining such determination methods. Examples of combinationswill be described later.

Next, the bad weather controller 151 included in the behavior plangenerator 150 will be described in detail. When the determiner 146 hasdetermined that the outside of the tunnel toward which the own-vehicle Mis traveling has bad weather, the bad weather controller 151 stopsautomated driving control and switches to manual driving control. Whenthe determiner 146 has determined that the outside of the tunnel towardwhich the own-vehicle M is traveling has bad weather, the bad weathercontroller 151 may perform control for decelerating the own-vehicle M.

For example, the bad weather controller 151 controls the brake device210 such that the speed after passing through the exit of the tunnelbecomes a predetermined speed or less.

When the determiner 146 has derived the certainty, the bad weathercontroller 151 may perform the above-described control according to thederived certainty. For example, the bad weather controller 151 does notperform the above-described control when the certainty is 0 and performsthe above-described control when the certainty is greater than 0. Whenthe certainty is greater than 0, the bad weather controller 151 maychange the amount of control and the control timing according to thecertainty. For example, when the certainty is low, the bad weathercontroller 151 delays the control timing or decreases the amount ofdeceleration as compared to when the certainty is high.

Next, an example of processing by the first controller 120 will bedescribed with reference to FIG. 9. FIG. 9 is a flowchart showing anexample of a flow of processing performed by the first controller 120.

First, the timing determiner 142 determines whether or not theown-vehicle M has reached a predetermined distance before the entranceof the tunnel (step S1). When the own-vehicle M has not reached thepredetermined distance before the entrance of the tunnel, the timingdeterminer 142 repeats the process until it is reached. When theown-vehicle M has reached the predetermined distance before the entranceof the tunnel, the state detector 144 performs a process for preparationat the entrance of the tunnel as necessary (step S3). This is performedwhen a state detection processing method which requires capturing of animage at the entrance is adopted. Details will be described later.

Next, the timing determiner 142 determines whether or not theown-vehicle M has reached a predetermined distance before the exit ofthe tunnel (step S5). When it is determined that the own-vehicle M hasnot reached the predetermined distance before the exit of the tunnel,the timing determiner 142 repeats the process until it is determinedthat it has been reached. When it is determined that the own-vehicle hasreached the predetermined distance before the exit of the tunnel, thestate detector 144 performs a state detection process (step S7). Then,on the basis of the detection result of the state detector 144, thedeterminer 146 performs a process of determining whether or not theoutside of the tunnel toward which the own-vehicle M is traveling hasbad weather (step S9).

Then, the bad weather controller 151 determines whether or not thedetermination result of the determiner 146 indicates that the weather isbad (step S11). When the determination result of the determiner 146indicates that the weather is bad, the bad weather controller 151 stopsthe automated driving control or decelerates the own-vehicle M (stepS13). When the determination result of the determiner 146 indicates thatthe weather is not bad, the bad weather controller 151 does not performany processing.

Next, an example of processing by the determiner 146 will be describedwith reference to FIGS. 10 to 14. FIGS. 10 to 14 are flowcharts showingexamples of flows of processing performed by the determiner 146.Hereinafter, first to fifth different determination processes will bedescribed with reference to the respective drawings. The state detector144 detects a state (the state of the oncoming vehicle m, the state ofthe oncoming lane, the state of the opposite side of the exit of thetunnel, or the like) through the process of any of FIGS. 10 to 14(corresponding to step S7 in FIG. 9). The determiner 146 determineswhether or not the weather is bad on the opposite side of the exit ofthe tunnel through the process of any of FIGS. 10 to 14 (correspondingto step S9 in FIG. 9).

First, the first determination process of the determiner 146 will bedescribed with reference to FIG. 10. Upon reaching a predetermineddistance before the entrance of the tunnel, the state detector 144acquires an image (for example, the image 304) obtained by imaging thesurroundings of the own-vehicle M near the entrance of the tunnel out ofimages captured by the camera 10 and derives an average luminance valueBv21 of the external image region 304 b on the basis of the acquiredimage 304 (step S101). This process corresponds to step S3 in FIG. 9.Next, upon reaching a predetermined distance before the exit of thetunnel, the state detector 144 detects an image (for example, the image305) obtained by imaging the surroundings of the own-vehicle M near theexit of the tunnel out of images captured by the camera 10 and derivesan average luminance value Bv22 of the external image region 305 a onthe basis of the acquired image 305 (step S103).

Then, the determiner 146 determines whether or not the differencebetween the average luminance value Bv21 derived in step S101 and theaverage luminance value Bv22 derived in step S103 is equal to or greaterthan a threshold value (step S105). When the difference between theaverage luminance values is equal to or greater than the thresholdvalue, the determiner 146 determines that the weather is bad outside theexit of the tunnel (step S107). On the other hand, when the differencebetween the average luminance values is not equal to or greater than thethreshold value, the determiner 146 determines that the weather is notbad outside the exit of the tunnel.

Next, the second determination process of the determiner 146 will bedescribed with reference to FIG. 11. Upon reaching a predetermineddistance before the exit of the tunnel, the state detector 144 detectsthe state of the oncoming vehicle m (step S201). Then, the determiner146 determines whether or not a state in which snow is attached to theoncoming vehicle m has been detected by the state detector 144 (stepS203).

Upon determining that a state in which snow is attached to the oncomingvehicle m has been detected, the determiner 146 determines that theweather is bad outside the exit of the tunnel (step S205). On the otherhand, upon determining that a state in which snow is attached to theoncoming vehicle m has not been detected, the determiner 146 determineswhether or not a state in which the oncoming vehicle m is wet has beendetected by the state detector 144 (step S207). Upon determining that astate in which the oncoming vehicle m is wet has been detected, thedeterminer 146 determines that the weather is bad outside the exit ofthe tunnel (step S205). On the other hand, upon determining that a statein which the oncoming vehicle m is wet has not been detected, thedeterminer 146 determines whether or not a state in which the wiper ofthe oncoming vehicle m is operating has been detected by the statedetector 144 (step S209). Upon determining that a state in which thewiper of the oncoming vehicle m is operating has been detected, thedeterminer 146 determines that the weather is bad outside the exit ofthe tunnel (step S205). On the other hand, upon determining that a statein which the wiper of the oncoming vehicle m is operating has not beendetected, the determiner 146 ends the process.

Next, the third determination process of the determiner 146 will bedescribed with reference to FIG. 12. The third determination process isan example of deriving the certainty on the basis of the state of theoncoming vehicle m. Processes similar to those in the seconddetermination process are denoted by the same reference numerals and adetailed description thereof will be omitted. Points P1 to P3 mentionedin the description are 0 in the initial state.

Upon determining in step S203 that a state in which snow is attached tothe oncoming vehicle m has been detected, the determiner 146 adds 1 tothe point P1 (step S206). Next, regardless of whether or not it isdetermined in step S203 that a state in which snow is attached to theoncoming vehicle m has been detected, the determiner 146 determineswhether or not a state in which the oncoming vehicle m is wet has beendetected (step S207). Upon determining that a state in which theoncoming vehicle m is wet has been detected, the determiner 146 adds 1to the point P2 (step S208). Next, regardless of whether or not it isdetermined in step S207 that a state in which the oncoming vehicle m iswet has been detected, the determiner 146 determines whether or not astate in which the wiper of the oncoming vehicle m is operating has beendetected (step S209). Upon determining that a state in which the wiperof the oncoming vehicle m is operating has been detected, the determiner146 adds 1 to the point P3 (step S210). Next, the determiner 146 sumsthe points P1 to P3 and outputs a total point (certainty) obtained bythe summation to the bad weather controller 151 (step S211).

Thereafter, the determiner 146 resets the values of the points P1 to P3.

When the certainty is derived by the determiner 146 as in the thirddetermination process, the bad weather controller 151 may determinewhether or not the certainty is 0, instead of the processing in step S5in FIG. 9, and may perform the process of step S7 in FIG. 9 upondetermining that the certainty is not 0. Upon determining that thecertainty is not 0, the bad weather controller 151 may perform controlaccording to the certainty instead of the process of step S7 in FIG. 9.

Next, the fourth determination process of the determiner 146 will bedescribed with reference to FIG. 13. Upon reaching a predetermineddistance before the exit of the tunnel, the state detector 144 detectsthe road surface state of the oncoming lane (step S301). Then, thedeterminer 146 determines whether or not a state in which the roadsurface of the oncoming lane is wet (or frozen) has been detected by thestate detector 144 (step S303). Upon determining that a state in whichthe road surface of the oncoming lane is wet has been detected, thedeterminer 146 determines that the weather is bad outside the exit ofthe tunnel (step S305). On the other hand, upon determining that a statein which the road surface of the oncoming lane is wet has not beendetected, the determiner 146 ends the process.

Next, the fifth determination process of the determiner 146 will bedescribed with reference to FIG. 14. The fifth determination process isa combination of the second determination process and the fourthdetermination process, and processes similar to those in the seconddetermination process are denoted by the same reference numerals and adetailed description thereof will be omitted.

Upon reaching a predetermined distance before the exit of the tunnel,the state detector 144 detects both the state of the oncoming vehicle mand the road surface state of the oncoming lane (step S202). Then, thedeterminer 146 performs at least one of the processes of steps S203,S207, and S209 on the basis of the detection result of the statedetector 144. Upon determining in step S203 that a state in which snowis attached to the oncoming vehicle m has been detected, upondetermining in step S207 that a state in which the oncoming vehicle m iswet has been detected, or upon determining in step S209 that a state inwhich the wiper of the oncoming vehicle m is operating has beendetected, the determiner 146 determines whether or not a state in whichthe road surface of the oncoming vehicle is wet (or frozen) has beendetected by the state detector 144 in step S202 (step S204). Upondetermining that a state in which the road surface of the oncoming laneis wet has been detected, the determiner 146 determines that the weatheris bad outside the exit of the tunnel (step S205). On the other hand,upon determining in step S204 that a state in which the road surface ofthe oncoming lane is wet has not been detected, and upon determining instep S209 that a state in which the wiper of the oncoming vehicle m isoperating has not been detected, the determiner 146 ends the process.Thereby, even when the oncoming vehicle m having snow attached theretoor the oncoming vehicle m that is wet has been detected or when theoncoming vehicle m on which the wiper is operating has been detected,the weather is more likely to be bad on the opposite side of the areanear the exit of the tunnel than the area near the exit of the tunnel ifthe road surface near the exit of the tunnel is not wet. In such a case,the determiner 146 determines that the place where snow or rain falls isnot the area near the exit of the tunnel and determines that the weatheris not bad on the opposite side of the area near the exit of the tunnel.

According to the vehicle control device of the present embodimentdescribed above, the state detector 144 that detects the state of theoncoming vehicle m facing the own-vehicle M in the tunnel, and thedeterminer 146 that determines whether or not the outside of the tunneltoward which the own-vehicle M is traveling has bad weather on the basisof the state of the oncoming vehicle m detected by the state detector144 are provided, whereby it is possible to acquire the weathercondition of the opposite side of the exit of the tunnel, regardless ofthe communication environment, and if the weather is bad on the oppositeside of the exit of the tunnel, it is possible to perform drivingcontrol according to bad weather. For example, by switching fromautomated driving control to manual driving control, it is possible tochange to careful driving by the driver. By decelerating the own-vehicleM, it is also possible to contribute to avoidance of accidents such asslipping.

Second Embodiment

An example in which a recognizer 130 and a bad weather controller 151having functions and configurations similar to those of the firstcontroller 120 described above are used for a vehicle having a drivingsupport function will be described below with reference to FIG. 15.

FIG. 15 is a configuration diagram of a vehicle system 1A that uses avehicle control device according to the embodiment for a vehicle havinga driving support function. Descriptions of functions and configurationssimilar to those of the vehicle system 1 will be omitted. For example,the vehicle system 1A includes a driving support control unit 300 inplace of some of the components of the vehicle system 1. The drivingsupport control unit 300 includes the recognizer 130 and a drivingsupport controller 310. The driving support controller 310 includes thebad weather controller 151. The components shown in FIG. 15 are merelyan example and some of the components may be omitted or other componentsmay be added.

For example, the driving support controller 310 has functions such as alane keeping assist system (LKAS), an adaptive cruise control system(ACC), and an auto lane change system (ALC). For example, under thecontrol of the bad weather controller 151, the driving supportcontroller 310 performs automatic deceleration control such that thespeed after passing through the exit of the tunnel becomes apredetermined speed or less. Under the control of the bad weathercontroller 151, the driving support controller 310 stops driving supportcontrol and switches to manual driving control.

According to the vehicle control device of the second embodimentdescribed above, it is possible to achieve the same advantages as thoseof the first embodiment.

<Hardware Configuration>

The vehicle control device of the embodiments described above isrealized, for example, by a hardware configuration as shown in FIG. 16.FIG. 16 is a diagram showing an example of the hardware configuration ofthe vehicle control device according to an embodiment.

The vehicle control device is configured such that a communicationcontroller 100-1, a CPU 100-2, a RAM 100-3, a ROM 100-4, a secondarystorage device 100-5 such as a flash memory or an HDD, and a drivedevice 100-6 are connected to each other via an internal bus or adedicated communication line. A portable storage medium such as anoptical disc is mounted in the drive device 100-6. A program 100-5 astored in the secondary storage device 100-5 is loaded in the RAM 100-3by a direct memory access (DMA) controller (not shown) or the like andthen executed by the CPU 100-2, thereby realizing the vehicle controldevice. The program referred to by the CPU 100-2 may be stored in theportable storage medium mounted in the drive device 100-6 or may bedownloaded from another device via a network NW.

The embodiments described above can be expressed as follows.

A vehicle control device includes:

a storage device; and

a hardware processor configured to execute a program stored in thestorage device,

wherein, by executing the program, the hardware processor is caused to:

detect a state of an external appearance of an oncoming vehicle facingan own-vehicle in a tunnel on the basis of an image captured by animaging unit configured to image surroundings of the own-vehicle; and

determine whether or not the state of the external appearance of theoncoming vehicle satisfies a predetermined condition on the basis of thedetected state of the external appearance of the oncoming vehicle, anddetermine that the outside of the tunnel toward which the own-vehicle istraveling has bad weather if the state of the external appearance of theoncoming vehicle satisfies the predetermined condition.

Although the modes for carrying out the present invention have beendescribed above by way of embodiments, the present invention is notlimited to these embodiments at all and various modifications andsubstitutions can be made without departing from the gist of the presentinvention.

For example, when the determiner 146 has determined that the weather isbad on the opposite side of the exit of the tunnel, the bad weathercontroller 151 may output this fact through the HMI 30 and notify theoccupant of this fact. When the determiner 146 has determined that theweather is bad on the opposite side of the exit of the tunnel, the badweather controller 151 may also output this fact through the HMI 30 andnotify the occupant of this fact before stopping automated drivingcontrol or driving support control. Thus, the driver can prepare formanual driving. The driver can also see the reason for deceleration ofthe own-vehicle M.

The detector described in the claims includes, for example, the statedetector 144 and may further include at least one of the camera 10, theradar device 12, the finder 14, and the object recognition device 16.

What is claimed is:
 1. A vehicle control device comprising: an imagingunit configured to image surroundings of an own-vehicle; a detectorconfigured to detect a state of an external appearance of an oncomingvehicle facing the own-vehicle in a tunnel on the basis of an imagecaptured by the imaging unit; and a determiner configured to determinewhether or not the state of the external appearance of the oncomingvehicle satisfies a predetermined condition on the basis of the state ofthe external appearance of the oncoming vehicle detected by the detectorand to determine that the outside of the tunnel toward which theown-vehicle is traveling has bad weather if the state of the externalappearance of the oncoming vehicle satisfies the predeterminedcondition.
 2. The vehicle control device according to claim 1, whereinthe determiner is configured to determine that the outside of the tunneltoward which the own-vehicle is traveling has bad weather if thedetector detects that snow is attached to the oncoming vehicle.
 3. Thevehicle control device according to claim 1, wherein the determiner isconfigured to determine that the outside of the tunnel toward which theown-vehicle is traveling has bad weather if the detector detects that awiper of the oncoming vehicle is operating.
 4. The vehicle controldevice according to claim 1, wherein the detector is configured tofurther detect a state of a road surface of an oncoming lane in whichthe oncoming vehicle is present on the basis of an image captured by theimaging unit, and the determiner is configured to determine that theoutside of the tunnel toward which the own-vehicle is traveling has badweather if the detector detects that the road surface of the oncominglane is wet.
 5. The vehicle control device according to claim 1, whereinthe determiner is configured to compare a luminance value of an imagecaptured near an entrance of the tunnel by the imaging unit and aluminance value of an image captured near an exit of the tunnel by theimaging unit and to determine that the outside of the tunnel towardwhich the own-vehicle is traveling has bad weather on the basis of aresult of the comparison.
 6. The vehicle control device according toclaim 1, further comprising a driving controller configured to controlone or both of steering or acceleration/deceleration of the own-vehicle,wherein the driving controller is configured to stop control if thedeterminer determines that the outside of the tunnel toward which theown-vehicle is traveling has bad weather.
 7. The vehicle control deviceaccording to claim 1, further comprising a driving controller configuredto control one or both of steering or acceleration/deceleration of theown-vehicle, wherein the driving controller is configured to deceleratethe own-vehicle if the determiner determines that the outside of thetunnel toward which the own-vehicle is traveling has bad weather.
 8. Avehicle control method performed by an in-vehicle computer mounted in anown-vehicle, the vehicle control method comprising: the in-vehiclecomputer detecting a state of an external appearance of an oncomingvehicle facing the own-vehicle in a tunnel on the basis of an imagecaptured by an imaging unit configured to image surroundings of theown-vehicle; determining whether or not the state of the externalappearance of the oncoming vehicle satisfies a predetermined conditionon the basis of the detected state of the external appearance of theoncoming vehicle; and determining that the outside of the tunnel towardwhich the own-vehicle is traveling has bad weather if the state of theexternal appearance of the oncoming vehicle satisfies the predeterminedcondition.
 9. A computer readable non-transitory storage medium storinga program causing an in-vehicle computer mounted in an own-vehiclehaving an imaging unit configured to image surroundings of theown-vehicle to: detect a state of an external appearance of an oncomingvehicle facing the own-vehicle in a tunnel on the basis of an imagecaptured by the imaging unit; determine whether or not the state of theexternal appearance of the oncoming vehicle satisfies a predeterminedcondition on the basis of the detected state of the external appearanceof the oncoming vehicle; and determine that the outside of the tunneltoward which the own-vehicle is traveling has bad weather if the stateof the external appearance of the oncoming vehicle satisfies thepredetermined condition.